For many years image generators have been a key component of simulation devices used to train operators such as airline pilots. The value of the training experience is highly dependent on the realism of the simulation. One aspect of simulation devices that has received much attention over the years is transport delay, or latency. Transport delay is defined as the time for a stimulus, such as the pilot moving the control stick, until the last pixel on the screen has been drawn 24, as shown in FIG. 1.
Many training simulators have a house sync 20 that is used to synchronize all of the simulation hardware. Each time the vehicle host computer receives a sync pulse, it will sample the current position of the controls and switches and compute the behavior of the vehicle. Upon completion, the updated positional information will be sent to the image generators so the display image can be updated. The time it takes to actually sample the controls and switches and then send this information to the image generator is the host delay 22. The image generator also has a display sync 26 which times and regulates the image generator functions.
One important area of delay is the delay of the image generator. The image generator's portion of this delay is defined as the time from when the image generator receives a position update from the host computer until the last pixel is drawn on a visual display which represents the new position 28.
Two aspects of transport delay are critical to training. The first aspect is determinism, or repeatable delay. The second important aspect is the length of the transport delay. If the transport delay does not remain constant, or if the delay is too long, the operator will often be overcome with simulator sickness.
There are two basic architectures in use today for simulation visual systems. One architecture provides a shorter transport delay than the other, but it is substantially more expensive and less deterministic than the other approach. Typical workstation visual systems consist of the major processes, as shown in FIG. 2A. A simulator host computer sends an eye position update to the image generator. The visual system has a real-time controller which receives this update and computes the transformation matrices and other parameters necessary to render an image from the new current position 10. Those transformation matrices are then applied to all of the potentially visible primitives (polygons, lines, dots, etc.) in the simulation database 12. Once transformed into screen space the primitives are loaded into the FIFO queue 13. Then the primitives can be rendered 14 into a pixel frame buffer memory 15, and displayed on the pilot's view screen 16.
This first basic architecture is a standard three-dimensional (3D) graphics computer or a workstation system which can be used to perform these operations. With such an architecture, the visual system's transport delay is illustrated in FIG. 3. The vertical arrows 30a, 30b, 30c represent the transfer of positional information from the host computer to the image generator. The dark shaded boxes represent the flow of one field of data through the graphics pipeline. The box indicates the amount of time allocated for each process, while the shaded portion indicates when the process is active. Usually the process will complete before the allocated time is up, indicated by the sloped right edge of the shaded portion. If the process takes longer than the available time, the system will be in an overload condition. The lighter shaded boxes show how adjacent fields are processed back to back.
The simulation host computer sends positional update information 30a-30c to the image generator once each display field. The real-time controller then computes the matrices and other information needed to display the scene 32. The real-time calculations begin as soon as the system receives input from the host (the black down arrow 30a). The real-time controller computes the eye position matrices, computes the position and orientation of moving models, updates embedded system behaviors, and then begins processing the database. This computation usually takes about ½ of a field time. The amount of time needed for this computation is dependent on the database, the current eye position, and the number of complex auxiliary functions and behaviors.
The geometry processing then begins on the primitives in the scene 34. As each primitive is transformed, it is handed to the rendering hardware 36. Specifically, the geometry processing begins storing processed polygons in its output FIFO queue as quickly as possible. Once the FIFOs contain data, the rendering engine can begin processing those primitives. As pixels are produced by the rendering engine, they are stored in a double buffered pixel memory while the previous field's data is being sent to the display for screen refresh. One full field time is allocated for this process, but it is important to complete both processes before the end of the field time or the system will be in an overload condition. Once the new image has been completed and written into one side of a double buffered pixel frame buffer, the buffer will be ready to toggle, or swap, at the next vertical sync pulse. After toggling, the new image is presented to the display device 38. Thus, the total transport delay for the visual system is 2.5 fields. As mentioned, standard image generator transport delay is measured from the input of host data (the down arrow 30a) to the display of the very last pixel on the screen (the right edge of the darkened display box 38).
Unfortunately, this approach has drawbacks that make it difficult to maintain deterministic behavior. Primitives cannot be rendered until after the geometric transformation operations are performed. The time required to find primitives and transform them is highly dependent on the database structure and the current position within the database. The time required to render primitives is directly related to their size on the screen. It seldom occurs that the geometry and rendering processes require the same amount of time, so one process usually ends up waiting for the other. This means that either the geometry engine or the rendering engine will sit idle during some portion of the field which reduces the efficiency of the system. Specifically, the FIFO between the geometry process and the rendering process cannot always guarantee optimum performance. If the rendering engine receives many small polygons, the FIFO may be drained faster than the geometry process can generate new polygons. This can cause the rendering process to be starved, and waste valuable rendering time. On the other hand, the rendering process may run too slowly on very large polygons, causing the FIFOs to fill up and the geometry process to stall. Furthermore, this loss of efficiency will often cause the system to overload since the entire job cannot be completed on time. The interactions between the geometry process and rendering process make load management more difficult since it is difficult to isolate which process is causing the overload condition. As a result, many systems need more geometry and rendering hardware than was originally expected, which increases the cost of the overall system. This non-deterministic characteristic makes this architecture less than an optimum choice for simulation applications. The efficiency of this system can be improved by using very large FIFOs and delaying the rendering operation until the FIFOs are sufficiently filled by the geometry operations to prevent the rendering process from running dry. This improves the efficiency, but unfortunately increases the transport delay.
Referring now to FIG. 2B, a second basic architecture has been used for many years in systems that are designed specifically for simulation. Another doubled buffered memory 18 (in addition to the double buffered pixel frame buffer) is inserted between the geometry and rendering processes to completely isolate them from each other. This memory stores primitives and is referred to as the geometry buffer. This gives one full field time for geometry calculations 40 and the next field time for pixel rendering 42 as illustrated in FIG. 4. Then the rendered image is displayed in the final field time 44. Of course, the processes are pipelined so each process is operating on something every frame and a new image is created each and every frame. The downside of this approach is obviously the increased transport delay caused by this additional buffer. As illustrated in FIG. 4, the transport time with this system is 3.5 fields.
This prior art process can also be illustrated in a flow diagram format, as shown in FIG. 5. The simulation host computer sends positional update information to the image generator 60 once each house sync (which is the same time interval as the display field time). The real-time controller then computes the matrices 62 and other information needed to display the scene. This computation usually takes about ½ of a field time 64. The geometry processing then begins 66 on the primitives in the scene. After all primitives have been transformed 68 and the previous field has been rendered 70 and the field timer is done 72, the geometry buffer is toggled 74 and primitives are handed to the rendering hardware 76. One full field time is allocated for each of these processes. Once the new image has been rendered and written into one side of a double buffered pixel frame buffer 78, the buffer will be ready to toggle, or swap, at the next vertical sync pulse 80. After toggling 82, the new image is presented to the display device 84.
The flow diagram of FIG. 5 follows one host input or house sync through the graphics pipeline. It should be noted that all processes actually occur every field time since a new input is received for each field time. Under normal operating conditions, the geometry process, the rendering process, and the display process (and thus the vertical sync pulse) all start at the same time. Furthermore, the geometry buffer and the pixel frame buffer toggle just prior to starting these processes.